WiHead: WiFi-Based Head-Pose Estimation

Due to the impact of Covid-19, people have started to conduct online courses or meetings. However, this makes it difficult to communicate with each other effectively because of the lack of non-verbal communication. Although webcams are available for online courses, etc., people often do not want to turn them on for privacy reasons. Thus, there is a need to develop privacy preserving way to enable non-verbal communication in online learning and work environments. WiFi as a sensor can be used to detect non-verbal gestures such as head poses, and has been increasingly valued due to its advantages of avoiding the effects of light, non-line of sight monitoring, privacy protection, etc. In this paper, we proposed an approach, which uses WiFi CSI data to estimate head pose. Our approach not only use the amplitude and phase data of raw CSI data, but also use the information in frequency domain. Our experiment with proposed approach confirmed the feasibility of head pose estimation based on WiFi CSI data. This has important implications for device-free sensing detection. Especially in today’s world where web conferences and online courses are widely used, WiFi-based head recognition can give feedback to the other party while protecting privacy, which helps to improve the quality and comfort of communication. © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.

Smart Artificial Intelligence Based Online Proctoring System

Since COVID 19, there have been significant advancements in the field of teaching and learning. Academic institutions are going digital to provide their students more resources. Due to technology, students now have more alternatives to study and improve skills at their own pace. In terms of assessments, there has been a shift toward online tests. The absence of a physical invigilator is perhaps the most significant impediment in online mode. Henceforth, online proctoring services are becoming more popular, and AI-powered proctoring solutions are becoming demanding. In this project, we describe a strategy for avoiding the physical presence of a proctor during the test by developing a multi-modal system. We captured video using a webcam along active window capture. The face of the test taker is identified and analyzed to forecast his emotions. To identify his head pose, his feature points are identified. Furthermore, aspects including a phone, a book, or the presence of another person are detected. This combination of models creates an intelligent rule-based inference system which is capable of determining if any malpractice took place during the examination. © 2022 IEEE.

Recognizing Students and Detecting Student Engagement with Real-Time Image Processing

With COVID-19, formal education was interrupted in all countries and the importance of distance learning has increased. It is possible to teach any lesson with various communication tools but it is difficult to know how far this lesson reaches to the students. In this study, it is aimed to monitor the students in a classroom or in front of the computer with a camera in real time, recognizing their faces, their head poses, and scoring their distraction to detect student engagement based on their head poses and Eye Aspect Ratios. Distraction was determined by associating the students’ attention with looking at the teacher or the camera in the right direction. The success of the face recognition and head pose estimation was tested by using the UPNA Head Pose Database and, as a result of the conducted tests, the most successful result in face recognition was obtained with the Local Binary Patterns method with a 98.95% recognition rate. In the classification of student engagement as Engaged and Not Engaged, support vector machine gave results with 72.4% accuracy. The developed system will be used to recognize and monitor students in the classroom or in front of the computer, and to determine the course flow autonomously.

Head Pose Estimation in Complex Environment Based on Four-branch Feature Selective Extraction and Regional Information Exchange Fusion Network

Under the severe situation of the COVID-19 pandemic, masks cover most of the effective facial features of users, and their head pose changes significantly in a complex environment, which makes the accuracy of head pose estimation in some systems such as safe driving systems and attention detection systems impossible to guarantee. To this end, we propose a powerful four-branch feature selective extraction network (FSEN) structure, in which three branches are used to extract three independent discriminative features of pose angles, and one branch is used to extract composite features corresponding to multiple pose angles. By reducing the dimension of high-dimensional features, our method significantly reduces the amount of computation while improving the estimation accuracy. Our convolution method is an improved spatial channel dynamic convolution (SCDC) that initially enhances the extracted features. Additionally, we embed a regional information exchange network (RIEN) after each convolutional layer in each branch to fully mine the potential semantic correlation between regions from multiple perspectives and learn and fuse this correlation to further enhance feature expression. Finally, we fuse the independent discriminative features of each pose angle and composite features from the three directions of channel, space, and pixel to obtain perfect feature expression for each pose angle, and then obtain the head pose angle. We conducted extensive experiments on the controlled environment datasets and a self-built real complex environment dataset (RCE) and the results showed that our method outperforms state-of-the-art single-modality methods and performs on par with multimodality-based methods. This shows that our network meets the requirements of accurate head-pose estimation in real complex environments such as complex illumination and partial occlusion. Author